Systems and methods for optical image stabilization

ABSTRACT

Systems and methods are disclosed for implementing optical image stabilization (OIS) in a mobile device. One or more functions associated with OIS may be performed by hardware and/or processing resources that are provided independently of a camera unit.

FIELD OF THE PRESENT DISCLOSURE

This disclosure generally relates to architectures for providing opticalimage stabilization and more specifically to efficient use of gyroscopicmotion sensor data in a mobile device.

BACKGROUND

Advances in technology have enabled the introduction of mobile devicesthat have feature an ever increasing set of capabilities. Smartphones,for example, now offer sophisticated computing and sensing resourcestogether with expanded communication functionality. Likewise, tablets,wearables, media players and other similar devices have shared in thisprogress. Notably, it is desirable and increasing common to provide amobile device with digital imaging functions. However, although anycamera may be subject to blurring as a consequence of movement when theimage is being captured, implementations in a mobile device may beparticularly susceptible due to consumer demand for such cameras to besmaller, cheaper and higher in resolution. A camera incorporated into amobile device is often hand held during use and, despite efforts to bestill during image recording, shaking inevitably occurs. Further,providing for increased exposure times is generally desirable, but tendsto exacerbate the effects of shaking as well.

An effective technique for minimizing blur in a digital image is opticalimage stabilization (OIS). Such systems sense motion associated withhand shake and provoke a compensating motion in either the camera lensor the image sensor to reduce blur. As will be appreciated by those ofskill in the art, using OIS may provide significantly improved imagequality, particularly at higher resolutions, longer exposure times orwhen employing an optical zoom.

Despite these advantages, OIS systems may require significant hardwareand processing resources. To facilitate incorporation into a mobiledevice, a digital camera featuring OIS may be integrated as a singleunit. Generally, such systems include the motion sensor, which may be atwo axis gyroscope, sensor processing to determine compensatory motionin response to the sensed motion, actuators to provide the compensatorymotion in the image sensor or lens, and position sensors to determinewhether the actuators have produced the desired movement.

Further, a conventionally integrated camera and OIS system may requirerelatively expensive and time consuming calibration. For example, eachcamera unit may be subjected to test vibrations during the manufacturingprocess in order to correct for mounting errors, gain errors and/orcross axis errors.

It is also noted that a typical manufacturing process for a digitalcamera involves cleaning the lens ultrasonically. Technologies used toimplement the motion sensor for an OIS system, such as amicroelectromechanical systems (MEMS) fabrication, may be damaged by thecleaning process. Accordingly, additional care and costs are associatedwith the manufacture of camera units featuring completely integrated OISsystems.

In light of the above, it would be desirable to facilitate theincorporation of a digital camera having OIS capabilities into a mobiledevice. Likewise, it would be desirable to do so while reducing the costand complexity of a camera unit. Still further, it would be desirable toreduce the burden of manufacturing and calibrating such mobile devices.As will be described in detail below, the techniques of this disclosuresatisfy these and other needs.

SUMMARY

As will be described in detail below, this disclosure relates to anoptical image stabilization (OIS) system including a first camera unitincluding a lens, an image sensor and an actuator for moving the lensrelative to the image sensor along at least two orthogonal axes, amotion sensor separate from the first camera unit including a gyroscopeconfigured to sense angular velocity of the first camera unit on atleast the two orthogonal axes, a motion processor configured todetermine movement of the first camera unit based at least in part onoutput from the motion sensor, an OIS controller configured to output asignal corresponding to a compensating relative movement between thelens and image sensor in response to the determined movement of thefirst camera unit, an actuator circuit for translating the output signalto actuator movement along the at least two orthogonal axes and adigital interface coupling the actuator circuit and the OIS controller.

In one aspect, the OIS controller and the actuator circuit may beimplemented in separate packages coupled by the digital interface,wherein the OIS controller and the motion sensor are implemented in thesame package. Alternatively. the OIS controller and the actuator circuitmay be implemented in a single package.

In one aspect, the OIS system may include a position sensor configuredto determine a position of the lens relative to the image sensor alongthe at least two orthogonal axes. Further, the position sensor mayprovide feedback to the actuator circuit. Additionally, the system mayinclude a position circuit configured to receive output from theposition sensor to determine the position of the lens relative to theimage sensor. In one embodiment, the position circuit, the actuatorcircuit and the OIS controller are implemented in a single package. Thesystem may also include a position controller to receive feedback fromthe position circuit.

In one aspect, the OIS system may include a host processor configured torun a process utilizing output from the motion sensor, wherein themotion processor simultaneously provides output to the host processorand the OIS controller. The process utilizing output from the motionsensor may be a user interface function.

In one aspect, the motion sensor further comprises an accelerometer. Themotion sensor may also include a magnetometer and a pressure sensor. TheOIS system may also include a sensor hub receiving input from at leastone additional sensor. Further, the output provided to the hostprocessor by the motion processor may be a result of a sensor fusionoperation.

In one aspect, the actuator may be configured to move the lens alongthree orthogonal axes and the motion sensor may sense motion along threeorthogonal axes, such that the OIS system may also include an autofocus(AF) controller configured to output a signal corresponding to movementalong at least one of the three orthogonal axes in response to thedetermined movement of the first camera unit. An image processorreceiving output from the image sensor may also provide focusinformation to the AF controller.

In one aspect, the OIS system may include an image processor receivingoutput from the image sensor, wherein information from the imageprocessor is used to assess the compensating relative movement betweenthe lens and image sensor.

In one aspect, the OIS system may include a second camera unit, whereinthe OIS controller is further configured to output a signalcorresponding to a compensating relative movement between a lens and animage sensor of the second camera unit in response to the determinedmovement.

This disclosure is also directed to methods for optically stabilizing animage. One suitable method may include providing a first camera unitincluding a lens, an image sensor and an actuator for moving the lensrelative to the image sensor along at least two orthogonal axes, sensingmotion of the first camera unit using a gyroscope separate from thefirst camera unit configured to sense angular velocity of the firstcamera unit on at least the two orthogonal axes, processing the sensedmotion to determine movement of the first camera unit, determining acompensating relative movement between the lens and the image sensor inresponse to the determined movement of the first camera unit andcontrolling the actuator to produce the compensating relative movementby sending an output signal from a processor that determines thecompensating relative movement to an actuator circuit for translatingthe output signal to actuator movement along the at least two orthogonalaxes over a digital interface.

In one aspect, a position of the lens relative to the image sensor alongthe at least two orthogonal axes may be determined using a positionsensor. The position sensor may provide feedback to the actuator circuitand a calibration routine may be performed using the feedback.

In one aspect, movement of the first camera unit may be determined andoutput to a host processor configured to run a process utilizing outputfrom the motion sensor may be provided simultaneously. The processutilizing output from the motion sensor may be a user interfacefunction. In one embodiment, the motion sensor may include anaccelerometer such that providing the output to the host processor bythe motion processor includes performing a sensor fusion operation.

In one aspect, the actuator may move the lens along three orthogonalaxes and the motion sensor may sense motion along three orthogonal axes,such that the method includes performing an autofocus (AF) process tooutput a signal corresponding to movement along at least one of thethree orthogonal axes in response to the determined movement of thefirst camera unit. Additionally, output may be received from the imagesensor with an image processor, so that the AF process includes focusinformation determined by the image processor.

In one aspect, output from the image sensor may be processed to assessthe compensating relative movement between the lens and image sensor.Additionally, a calibration routine may be performed using theassessment of the compensating relative movement.

In one aspect, the motion sensor and the actuator may be calibratedindependently.

In one aspect, a second camera unit having a lens, an image sensor andan actuator for moving the lens relative to the image sensor along atleast two orthogonal axes may be provided, so that the method includesdetermining a compensating relative movement between the lens and theimage sensor of the second camera unit and controlling the actuator ofthe second camera

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a device for providing OIS of a digitalimage according to an embodiment.

FIG. 2 is a schematic diagram of a device for providing OIS featuring anintegrated motion sensor and OIS controller separate from a camera unitaccording to an embodiment.

FIG. 3 is a schematic diagram of a device for providing OIS featuring anintegrated motion sensor, OIS controller, actuator circuit and positioncircuit separate from a camera unit according to an embodiment.

FIG. 4 is a flowchart showing a routine for providing OIS of a digitalimage according to an embodiment.

DETAILED DESCRIPTION

At the outset, it is to be understood that this disclosure is notlimited to particularly exemplified materials, architectures, routines,methods or structures as such may vary. Thus, although a number of suchoptions, similar or equivalent to those described herein, can be used inthe practice or embodiments of this disclosure, the preferred materialsand methods are described herein.

It is also to be understood that the terminology used herein is for thepurpose of describing particular embodiments of this disclosure only andis not intended to be limiting.

The detailed description set forth below in connection with the appendeddrawings is intended as a description of exemplary embodiments of thepresent disclosure and is not intended to represent the only exemplaryembodiments in which the present disclosure can be practiced. The term“exemplary” used throughout this description means “serving as anexample, instance, or illustration,” and should not necessarily beconstrued as preferred or advantageous over other exemplary embodiments.The detailed description includes specific details for the purpose ofproviding a thorough understanding of the exemplary embodiments of thespecification. It will be apparent to those skilled in the art that theexemplary embodiments of the specification may be practiced withoutthese specific details. In some instances, well known structures anddevices are shown in block diagram form in order to avoid obscuring thenovelty of the exemplary embodiments presented herein.

For purposes of convenience and clarity only, directional terms, such astop, bottom, left, right, up, down, over, above, below, beneath, rear,back, and front, may be used with respect to the accompanying drawingsor chip embodiments. These and similar directional terms should not beconstrued to limit the scope of the disclosure in any manner.

In this specification and in the claims, it will be understood that whenan element is referred to as being “connected to” or “coupled to”another element, it can be directly connected or coupled to the otherelement or intervening elements may be present. In contrast, when anelement is referred to as being “directly connected to” or “directlycoupled to” another element, there are no intervening elements present.

Some portions of the detailed descriptions which follow are presented interms of procedures, logic blocks, processing and other symbolicrepresentations of operations on data bits within a computer memory.These descriptions and representations are the means used by thoseskilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. In the presentapplication, a procedure, logic block, process, or the like, isconceived to be a self-consistent sequence of steps or instructionsleading to a desired result. The steps are those requiring physicalmanipulations of physical quantities. Usually, although not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated in a computer system.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the present application,discussions utilizing the terms such as “accessing,” “receiving,”“sending,” “using,” “selecting,” “determining,” “normalizing,”“multiplying,” “averaging,” “monitoring,” “comparing,” “applying,”“updating,” “measuring,” “deriving” or the like, refer to the actionsand processes of a computer system, or similar electronic computingdevice, that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices.

Embodiments described herein may be discussed in the general context ofprocessor-executable instructions residing on some form ofnon-transitory processor-readable medium, such as program modules,executed by one or more computers or other devices. Generally, programmodules include routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. The functionality of the program modules may becombined or distributed as desired in various embodiments.

In the figures, a single block may be described as performing a functionor functions; however, in actual practice, the function or functionsperformed by that block may be performed in a single component or acrossmultiple components, and/or may be performed using hardware, usingsoftware, or using a combination of hardware and software. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, circuits, and steps have beendescribed above generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure. Also, the exemplary wirelesscommunications devices may include components other than those shown,including well-known components such as a processor, memory and thelike.

The techniques described herein may be implemented in hardware,software, firmware, or any combination thereof, unless specificallydescribed as being implemented in a specific manner. Any featuresdescribed as modules or components may also be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a non-transitory processor-readable storagemedium comprising instructions that, when executed, performs one or moreof the methods described above. The non-transitory processor-readabledata storage medium may form part of a computer program product, whichmay include packaging materials.

The non-transitory processor-readable storage medium may comprise randomaccess memory (RAM) such as synchronous dynamic random access memory(SDRAM), read only memory (ROM), non-volatile random access memory(NVRAM), electrically erasable programmable read-only memory (EEPROM),FLASH memory, other known storage media, and the like. The techniquesadditionally, or alternatively, may be realized at least in part by aprocessor-readable communication medium that carries or communicatescode in the form of instructions or data structures and that can beaccessed, read, and/or executed by a computer or other processor. Forexample, a carrier wave may be employed to carry computer-readableelectronic data such as those used in transmitting and receivingelectronic mail or in accessing a network such as the Internet or alocal area network (LAN). Of course, many modifications may be made tothis configuration without departing from the scope or spirit of theclaimed subject matter.

The various illustrative logical blocks, modules, circuits andinstructions described in connection with the embodiments disclosedherein may be executed by one or more processors, such as one or moremotion processing units (MPUs), digital signal processors (DSPs),general purpose microprocessors, application specific integratedcircuits (ASICs), application specific instruction set processors(ASIPs), field programmable gate arrays (FPGAs), or other equivalentintegrated or discrete logic circuitry. The term “processor,” as usedherein may refer to any of the foregoing structure or any otherstructure suitable for implementation of the techniques describedherein. In addition, in some aspects, the functionality described hereinmay be provided within dedicated software modules or hardware modulesconfigured as described herein. Also, the techniques could be fullyimplemented in one or more circuits or logic elements. A general purposeprocessor may be a microprocessor, but in the alternative, the processormay be any conventional processor, controller, microcontroller, or statemachine. A processor may also be implemented as a combination ofcomputing devices, e.g., a combination of an MPU and a microprocessor, aplurality of microprocessors, one or more microprocessors in conjunctionwith an MPU core, or any other such configuration.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one having ordinaryskill in the art to which the disclosure pertains.

Finally, as used in this specification and the appended claims, thesingular forms “a, “an” and “the” include plural referents unless thecontent clearly dictates otherwise.

As noted above, it is increasingly desirable to provide a mobileelectronic device with one or more digital cameras. Correspondingly, itis also desirable to provide OIS systems to improve the image qualityproduced by such cameras. This disclosure is directed to reducing thecosts and complexities associated with conventional OIS implementations.In particular, one or more functions associated with OIS may beperformed by hardware and/or processing resources that are providedindependently of a camera unit. For example, a mobile device may have agyroscope and/or other suitable motion sensor that performs functionsunrelated to OIS. Notably, a mobile device may employ motion sensors aspart of the user interface, such as for determining orientation of thedevice to adjust the display of information accordingly as well as forreceiving user input for controlling an application, for navigationalpurposes, or for a wide variety of other applications. Data from such asensor may be used to determine motion of the mobile device for OIS sothat the camera unit does not require a dedicated motion sensor. As willbe appreciated, the user interface functions may not be required duringimage recording or the motion sensor may be able to perform bothfunctions (as well as others) simultaneously. Further, a processor orprocessing resources utilized for other functions in the mobile device,such as processing the sensor data, may be employed to perform tasksassociated with OIS, reducing or removing the need to provide dedicatedOIS processing on the camera unit. Such architecture designs allow for asimplified camera unit, as well as facilitating manufacture andcalibration.

Details regarding one embodiment of a mobile electronic device 100including features of this disclosure are depicted as high levelschematic blocks in FIG. 1. As will be appreciated, device 100 may beimplemented as a device or apparatus, such as a handheld device or adevice that is secured to a user. In one embodiment, device 100 may beconfigured as a smartphone as discussed in further detail below.However, the device may also be a tablet, laptop, personal digitalassistant (PDA), video game player, video game controller, navigationdevice, mobile internet device (MID), personal navigation device (PND),portable music, video, or media player, remote control, or otherhandheld device, or a combination of one or more of these devices.

As shown, device 100 includes a camera unit 102, incorporating lens 104,image sensor 106, actuator 108 for imparting relative movement betweenlens 104 and image sensor 106 along at least two orthogonal axes, andposition sensor 110 for determining the position of lens 104 in relationto image sensor 106. In one aspect, actuator 108 may be implementedusing voice coil motors (VCM) and position sensor 110 may be implementedwith Hall sensors, although other suitable alternatives may be employed.Device 100 may also include a host processor 112, memory 114, interfacedevices 116 and display 118. Host processor 112 can be one or moremicroprocessors, central processing units (CPUs), or other processorswhich run software programs, which may be stored in memory 114,associated with the functions of device 100. Interface devices 16 can beany of a variety of different devices providing input and/or output to auser, such as audio speakers, buttons, touch screen, joystick, slider,knob, printer, scanner, computer network I/O device, other connectedperipherals and the like. Display 118 may be configured to output imagesviewable by the user and may function as a viewfinder for camera unit102. Further, the embodiment shown features dedicated image processor120 for receiving output from image sensor 106, although in otherembodiments this functionality may be performed by host processor 112 orother processing resources.

Accordingly, multiple layers of software can be provided in memory 114,which may be any combination of computer readable medium such aselectronic memory or other storage medium such as hard disk, opticaldisk, etc., for use with the host processor 112. For example, anoperating system layer can be provided for device 100 to control andmanage system resources in real time, enable functions of applicationsoftware and other layers, and interface application programs with othersoftware and functions of device 100. Similarly, different softwareapplication programs such as menu navigation software, games, camerafunction control, navigation software, communications software, such astelephony or wireless local area network (WLAN) software, or any of awide variety of other software and functional interfaces can beprovided. In some embodiments, multiple different applications can beprovided on a single device 100, and in some of those embodiments,multiple applications can run simultaneously.

Device 100 also includes integrated motion processing unit (MPU™) 122featuring sensor processor 124, memory 126 and motion sensor 128. Memory126 may store algorithms, routines or other instructions for processingdata output by motion sensor 128 and/or other sensors as described belowusing logic or controllers of sensor processor 124, as well as storingraw data and/or motion data output by motion sensor 128 or othersensors. Motion sensor 128 may be one or more sensors for measuringmotion of device 100 in space. Depending on the configuration, MPU 122measures one or more axes of rotation and/or one or more axes ofacceleration of the device. In one embodiment, at least some of themotion sensors are inertial sensors, such as rotational motion sensorsor linear motion sensors. For example, the rotational motion sensors maybe gyroscopes to measure angular velocity along one or more orthogonalaxes and the linear motion sensors may be accelerometers to measurelinear acceleration along one or more orthogonal axes. In one aspect,three gyroscopes and three accelerometers may be employed, such that asensor fusion operation performed by sensor processor 124 or otherprocessing resources of device 100 provides a six axis determination ofmotion. As desired, motion sensor 128 may be implemented using MEMS tobe integrated with MPU 122 in a single package. Exemplary detailsregarding suitable configurations of host processor 112 and MPU 122 maybe found in co-pending, commonly owned U.S. patent application Ser. No.11/774,488, filed Jul. 6, 2007, and Ser. No. 12/106,921, filed Apr. 21,2008, which are hereby incorporated by reference in their entirety.Further, MPU 122 may be configured as a sensor hub by aggregating sensordata from additional processing layers as described in co-pending,commonly owned U.S. patent application Ser. No. 14/480,364, filed Sep.8, 2014, which is also hereby incorporated by reference in its entirety.Suitable implementations for MPU 122 in device 100 are available fromInvenSense, Inc. of Sunnyvale, Calif. Thus, MPU 122 is configured toprovide motion data for purposes independent of camera unit 102, such asto host processor 112 for user interface functions, as well as enablingOIS functionality.

Device 100 may also include other sensors as desired. As shown, analogsensor 130 may provide output to analog to digital converter (ADC) 132within MPU 122. Alternatively or in addition, data output by digitalsensor 134 may be communicated over digital bus 136 to sensor processor124 or other processing resources in device 100. Analog sensor 130 anddigital sensor 134 may provide additional sensor data about theenvironment surrounding device 100. For example, sensors such as one ormore pressure sensors, magnetometers, temperature sensors, infraredsensors, ultrasonic sensors, radio frequency sensors, or other types ofsensors can be provided. In one embodiment, data from a magnetometermeasuring along three orthogonal axes may be fused with gyroscope andaccelerometer data to provide a nine axis determination of motion.Further, a pressure sensor may be used as an indication of altitude fordevice 100, such that a sensor fusion operation may provide a ten axisdetermination of motion. In the context of the OIS techniques of thisdisclosure, any combination of sensors, including motion sensor 128,analog sensor 130 and digital sensor 134, all of which may beimplemented independently of camera unit 102, may be used to determineangular velocity of device 100 along the two orthogonal axes associatedwith the plane of image sensor 106.

In the embodiment shown, camera unit 102, MPU 122, host processor 112,memory 114 and other components of device 100 may be coupled throughdigital bus 136, which may be any suitable bus or interface, such as aperipheral component interconnect express (PCIe) bus, a universal serialbus (USB), a universal asynchronous receiver/transmitter (UART) serialbus, a suitable advanced microcontroller bus architecture (AMBA)interface, an Inter-Integrated Circuit (I2C) bus, a serial digital inputoutput (SDIO) bus, a serial peripheral interface (SPI) or otherequivalent. Depending on the architecture, different bus configurationsmay be employed as desired. For example, additional buses may be used tocouple the various components of device 100, such as by using adedicated bus between host processor 112 and memory 114.

As noted above, multiple layers of software may be employed as desiredand stored in any combination of memory 114, memory 126, or othersuitable location. For example, a motion algorithm layer can providemotion algorithms that provide lower-level processing for raw sensordata provided from the motion sensors and other sensors. A sensor devicedriver layer may provide a software interface to the hardware sensors ofdevice 100. Further, a suitable application program interface (API) maybe provided to facilitate communication between host processor 112 andMPU 122, for example, to transmit desired sensor processing tasks. Otherembodiments may feature any desired division of processing between MPU122 and host processor 112 as appropriate for the applications and/orhardware being employed. For example, lower level software layers may beprovided in MPU 122 and an API layer implemented by host processor 112may allow communication of the states of application programs as well assensor commands. Some embodiments of API implementations in a motiondetecting device are described in co-pending U.S. patent applicationSer. No. 12/106,921, incorporated by reference above.

Additionally, device 100 may include a plurality of digital imagingmodules, each of which may implement OIS utilizing general purposemotion sensing and/or processing capabilities according to thetechniques of this disclosure. For example, device 100 is shown withauxiliary camera unit 138. Although not shown for the purposes ofclarity, one of skill in the art will appreciate that auxiliary cameraunit 138 may include sufficient assemblies for OIS, such as actuators,position sensors and the like as described in the context of camera unit102 or the other camera units of this disclosure. In one embodiment,device 100 may be a smartphone, camera unit 102 may be configured as arear-facing camera and auxiliary camera unit 138 may be configured as afront-facing camera. In other embodiments, any suitable number of cameraunits may utilize the motion sensing capabilities of device 100 toimplement OIS.

In the described embodiments, a chip is defined to include at least onesubstrate typically formed from a semiconductor material. A single chipmay be formed from multiple substrates, where the substrates aremechanically bonded to preserve the functionality. A multiple chipincludes at least two substrates, wherein the two substrates areelectrically connected, but do not require mechanical bonding. A packageprovides electrical connection between the bond pads on the chip to ametal lead that can be soldered to a PCB. A package typically comprisesa substrate and a cover. Integrated Circuit (IC) substrate may refer toa silicon substrate with electrical circuits, typically CMOS circuits.MEMS cap provides mechanical support for the MEMS structure. The MEMSstructural layer is attached to the MEMS cap. The MEMS cap is alsoreferred to as handle substrate or handle wafer. In the describedembodiments, an MPU may incorporate the sensor. The sensor or sensorsmay be formed on a first substrate. Other embodiments may includesolid-state sensors or any other type of sensors. The electroniccircuits in the MPU receive measurement outputs from the one or moresensors. In some embodiments, the electronic circuits process the sensordata. The electronic circuits may be implemented on a second siliconsubstrate. In some embodiments, the first substrate may be verticallystacked, attached and electrically connected to the second substrate ina single semiconductor chip, while in other embodiments the firstsubstrate may be disposed laterally and electrically connected to thesecond substrate in a single semiconductor package.

As one example, the first substrate may be attached to the secondsubstrate through wafer bonding, as described in commonly owned U.S.Pat. No. 7,104,129, which is incorporated herein by reference in itsentirety, to simultaneously provide electrical connections andhermetically seal the MEMS devices. This fabrication techniqueadvantageously enables technology that allows for the design andmanufacture of high performance, multi-axis, inertial sensors in a verysmall and economical package. Integration at the wafer-level minimizesparasitic capacitances, allowing for improved signal-to-noise relativeto a discrete solution. Such integration at the wafer-level also enablesthe incorporation of a rich feature set which minimizes the need forexternal amplification.

In the described embodiments, raw data refers to measurement outputsfrom the sensors which are not yet processed. Depending on the context,motion data may refer to processed raw data, which may involve applyinga sensor fusion algorithm or applying any other algorithm. In the caseof a sensor fusion algorithm, data from one or more sensors may becombined to provide an orientation of the device. In the describedembodiments, an MPU may include processors, memory, control logic andsensors among structures.

Turning to FIG. 2, one suitable configuration of mobile device 200 isshown schematically in reference to communications between camera unit202 and MPU 212. Similar to the embodiment shown in FIG. 1, camera unit202 includes lens 204, image sensor 206, actuator 208 for impartingrelative movement between lens 204 and image sensor 206 along at leasttwo orthogonal axes, and position sensor 210 for determining theposition of lens 204 in relation to image sensor 206. As shown, MPU 212includes motion sensor 214, such as gyroscopes to measure angularvelocity of device 200 along at least two orthogonal axes correspondingto the plane of image sensor 206. MPU 212 may also receive motion sensordata from other integrated or external sensors as described above, andmay perform a sensor fusion operation to provide refined motion sensing,such as in the form of a six axis or nine axis motion determination. MPU212 also includes OIS controller 216 to determine compensating relativemotion between lens 204 and image sensor 206 to counter act shaking orother movement detected by motion sensor 214 and reduce blur in imagesrecorded by image sensor 206. For example, OIS controller 216 may beimplemented using the equivalent of sensor processor 124 or by othersuitable processing resources. In this embodiment, MPU 212 may beconfigured as a single die or may be two dies stacked into the samepackage.

In one aspect, OIS controller 216 may output a signal representing anamount of relative movement between lens 204 and image sensor 206 tocompensate for motion detected by motion sensor 214. The signal, such asa digital control word corresponding to the desired movement, may besent over OIS digital interface 218 to position controller 220 in cameraunit 202. In turn, position controller 220 may send the desired movementto actuator circuit 222 to be translated into the appropriate controlsignals for actuator 208, such as by converting the desired movementinto corresponding movements in each orthogonal axis of operation foractuator 208. As also shown, the data provided by position sensor 210 isprocessed by position circuit 224 to determine the actual relativemovement between image sensor 206 and lens 204 produced by actuator 208.This determination may be provided as feedback to position controller220 to help ensure that actuator 208 is operated in a manner thatproduces the desired amount of movement.

In another aspect, image sensor 206 is depicted as outputting data toimage processor 226 over camera digital interface 228. Further, imageprocessor 226 may be coupled to MPU 212 using host digital interface230. By utilizing feedback from image processor 226, OIS controller 216may refine the signal used to indicate the desired movement to actuator208. For example, image processor 226 may determine pixel shift as knownin the art to assess the degree to which the compensating movementdetermined by OIS controller 216 has succeeded in reducing blur. Thisinformation may be used in the calibration process of motion sensor 214,as well as in the calibration process of the actuator 208 and positionsensor 210 loop mediated by position controller 220. In someembodiments, feedback from image processor 226 may obviate the need forposition sensor 210 and position circuit 224. For example, a sufficientreduction in pixel shift as determined by image processor 226 may betaken as an indication that actuator 208 has produced the desiredrelative movement between image sensor 206 and lens 204 withoutrequiring the measurement provided by position sensor 210. Althoughshown as a dedicated processor, image processor 226 may also beimplemented using any suitable processing resources of device 100. Forexample, the functionality of image processor 226 may be provided by anapplication processor, such as host processor 112, a processor in MPU212 or a processor in camera unit 202.

As will be appreciated from the architecture shown in FIG. 2, separatingthe motion sensing capabilities from camera unit 202 enables a number ofsuitable calibration strategies. Actuator 208 and/or position sensor 210and their associated circuitry may be calibrated independently of motionsensor 214. A synthetic test signal may be provided to positioncontroller 220 to determine suitable offsets, sensitivity or gainadjustments, cross axis compensations, or any other calibrationparameters for actuator 208 and/or position sensor 210. Such values maybe determined during manufacture and stored in one-time programmable(OTP) memory 232 or other suitable non-volatile storage means. Forexample, a digital control word representing a defined movement may besent to position controller 220 to determine any necessary adjustment sothat actuator 208 responds with the defined movement. Alternatively orin addition, a reference signal provided by a single motion sensorhaving known characteristics may be supplied to a plurality of cameraunits to perform the calibration process. Such techniques may reduce oreliminate the need to perform a conventional vibration process involvingeach device under test being shaken during manufacture as part of thecalibration. Feedback from image processor 226 may also be used for runtime calibration as desired. For example, proper OIS operation may beconfirmed and suitable adjustments made based on a determination byimage processor 226 of pixel shift in the output of image sensor 206.Notably, calibration parameters determined from image processor 226 maybe aggregated during run time to provide look up reference duringsubsequent operations. In one aspect, the techniques of this disclosuremay be facilitated by separately calibrating motion sensor 214 to adesired level. For example, a gyroscope may be calibrated to asensitivity of approximately 1% or less.

In a further aspect, camera unit 202 may also offer auto focus (AF)capabilities. Actuator 208 may be configured to produce motion in threeorthogonal axes. As described above, relative motion in the twoorthogonal axes associated with the plane of image sensor 206 may beemployed for OIS purposes while relative motion in the third orthogonalaxis may focus the image on image sensor 206. Similarly, position sensor210 may also be configured to measure relative position along all threeorthogonal axes to provide feedback for the AF system as well as the OISsystem. AF controller 234 may be implemented as part of image processor226 to provide a focusing signal as known in the art. In one embodiment,the focusing signal may be sent to MPU 212 over interface 230 so that itmay be incorporated into the signal provided by OIS controller 216. Suchan embodiment allows a single combined positional signal reflecting bothAF and OIS adjustments to be sent to position controller 220. SuitableAF adjustments may be made by employing motion data determined by MPU212.

In yet another aspect, device 200 may include temperature sensor 236.Although depicted as being implemented in camera unit 202, thetemperature sensor may be provided by as another sensor, such as analogsensor 130 or digital sensor 134. As desired, operation of actuator 208and/or position sensor 210 may be adjusted depending on the measurementsof temperature sensor 236.

The embodiment shown in FIG. 2 shows three digital interfaces, OISdigital interface 218, camera digital interface 228 and host digitalinterface 230. Depending on the configuration, these interfaces may beseparate, may be a single interface such as bus 136 in FIG. 1, or may beany combination of these implementations. For example, as describedabove, feedback from AF controller 234 may be provided to MPU 212. Inother embodiments, feedback from AF controller 234 may be delivereddirectly to camera unit 202 to allow for AF operation without invokingOIS controller 216. Correspondingly, MPU 212 may be operated in a powersave mode of operation, while still allowing AF capabilities for cameraunit 202. In another aspect, OIS digital interface 218 may be configuredto utilize a low latency communication protocol to facilitate rapidadjustments to actuator 208 to compensate for the detected shaking ofdevice 200. For example, pulse width modulation or other similartechniques may be used.

Another embodiment of this disclosure is schematically depicted in FIG.3 in reference to mobile device 300. As in other embodiments, cameraunit 302 may also include lens 304, image sensor 306, actuator 308 andposition sensor 310. In a separate module, MPU 312 includes motionsensor 314 and OIS controller 316. Each of these components may functionin a similar manner to the corresponding elements described above. Inthis embodiment, MPU 312 implements actuator circuit 318 and positioncircuit 320 in a single package to reduce the complexity and cost ofcamera unit 302. Other combinations of the architectures represented byFIGS. 2 and 3 may also be used, such as by incorporating one of theposition circuit and the actuator circuit in the MPU and the other inthe camera unit. As shown, output from position sensor 310 may becommunicated over position interface 322 to ADC 324. In one aspect, ADC324 may be used for another purpose in MPU 312, as in the example of ADC132 shown in FIG. 1. Actuator circuit 318 may receive a signal from OIScontroller 316 corresponding to the desired relative movement betweenlens 304 and image sensor 306 to compensate for shaking detected bymotion sensor 314. Accordingly, actuator circuit 318 may separate thedesired movement into corresponding motion of actuator 308 along each oftwo orthogonal axes, as communicated over actuator interface 326. Ifdesired, position interface 322 and actuator interface 326 may becombined. In the embodiment shown, actuator circuit 318, positioncircuit 320 and OIS controller 316 may coupled by internal digital bus328, although other suitable configurations may be employed. Imagesensor 306 may transmit data to image processor 330 over camera digitalinterface 332, which may in turn be coupled to MPU 312 with host digitalinterface 334. As described above, OTP 336 may be used for storingsuitable calibration values determined during manufacture.

To help illustrate aspects of this disclosure, FIG. 4 is a flow chartshowing a suitable process providing OIS in a mobile device with adigital camera unit. As indicated by 400, a first camera unit, such ascamera unit 202, may be provided with a lens, an image sensor and anactuator for moving the lens relative to the image sensor along at leasttwo orthogonal axes. In 402, a motion sensor such as motion sensor 214may be used to sense motion of the first camera unit. Since motionsensor 214 and camera unit 202 are both part of mobile device 200,movement of motion sensor 214 may be taken as an indication of movementof camera unit 202. In 404, the sensed motion may be processed todetermine movement of the first camera unit, such as by MPU 212. Next, acompensating relative movement between the lens and the image sensor maybe determined in response to the determined movement of the first cameraunit, such as by OIS controller 216, in 406. The actuator may becontrolled in 408 to produce the compensating relative movement bysending an output signal from OIS controller to an actuator circuit fortranslating the output signal to actuator movement along the at leasttwo orthogonal axes over a digital interface, such as actuator circuit222.

Although the present invention has been described in accordance with theembodiments shown, one of ordinary skill in the art will readilyrecognize that there could be variations to the embodiments and thosevariations would be within the spirit and scope of the presentinvention. Accordingly, many modifications may be made by one ofordinary skill in the art without departing from the spirit and scope ofthe present invention.

What is claimed is:
 1. An optical image stabilization (OIS) systemcomprising: a first camera unit including a lens, an image sensor and anactuator for moving the lens relative to the image sensor along at leasttwo orthogonal axes; a motion sensor separate from the first camera unitincluding a gyroscope configured to sense angular velocity of the firstcamera unit on at least the two orthogonal axes; a motion processorconfigured to determine movement of the first camera unit based at leastin part on output from the motion sensor; an OIS controller configuredto output a signal corresponding to a compensating relative movementbetween the lens and image sensor in response to the determined movementof the first camera unit; an actuator circuit for translating the outputsignal to actuator movement along the at least two orthogonal axes; anda digital interface coupling the actuator circuit and the OIScontroller.
 2. The OIS system of claim 1, wherein the OIS controller andthe actuator circuit are implemented in separate packages coupled by thedigital interface.
 3. The OIS system of claim 2, wherein the OIScontroller and the motion sensor are implemented in the same package. 4.The OIS system of claim 1, wherein the OIS controller and the actuatorcircuit are implemented in a single package.
 5. The OIS system of claim1, further comprising a position sensor configured to determine aposition of the lens relative to the image sensor along the at least twoorthogonal axes.
 6. The OIS system of claim 5, wherein the positionsensor provides feedback to the actuator circuit.
 7. The OIS system ofclaim 6, further comprising a position circuit configured to receiveoutput from the position sensor to determine the position of the lensrelative to the image sensor.
 8. The OIS system of claim 7, wherein theposition circuit, the actuator circuit and the OIS controller areimplemented in a single package.
 9. The OIS system of claim 7, furthercomprising a position controller receiving feedback from the positioncircuit.
 10. The OIS system of claim 1, further comprising a hostprocessor configured to run a process utilizing output from the motionsensor, wherein the motion processor is configured to simultaneouslyprovide output to the host processor and the OIS controller.
 11. The OISsystem of claim 10, wherein the process utilizing output from the motionsensor comprises a user interface function.
 12. The OIS system of claim1, wherein the motion sensor further comprises an accelerometer.
 13. TheOIS system of claim 12, wherein the motion sensor further comprises amagnetometer and a pressure sensor.
 14. The OIS system of claim 12,wherein the output provided to the host processor by the motionprocessor comprises a result of a sensor fusion operation.
 15. The OISsystem of claim 1, further comprising a sensor hub receiving input fromat least one additional sensor.
 16. The OIS system of claim 1, whereinthe actuator moves the lens along three orthogonal axes and wherein themotion sensor senses motion along three orthogonal axes, furthercomprising an autofocus (AF) controller configured to output a signalcorresponding to movement along at least one of the three orthogonalaxes in response to the determined movement of the first camera unit.17. The OIS system of claim 16, further comprising an image processorreceiving output from the image sensor, wherein the image processorprovides focus information to the AF controller.
 18. The OIS system ofclaim 1, further comprising an image processor receiving output from theimage sensor, wherein information from the image processor is used toassess the compensating relative movement between the lens and imagesensor.
 19. The OIS system of claim 1, further comprising a secondcamera unit, wherein the OIS controller is further configured to outputa signal corresponding to a compensating relative movement between alens and an image sensor of the second camera unit in response to thedetermined movement.
 20. A method for optically stabilizing an image,comprising: providing a first camera unit including a lens, an imagesensor and an actuator for moving the lens relative to the image sensoralong at least two orthogonal axes; sensing motion of the first cameraunit using a gyroscope separate from the first camera unit configured tosense angular velocity of the first camera unit on at least the twoorthogonal axes; processing the sensed motion to determine movement ofthe first camera unit; determining a compensating relative movementbetween the lens and the image sensor in response to the determinedmovement of the first camera unit; and controlling the actuator toproduce the compensating relative movement by sending an output signalfrom a processor that determines the compensating relative movement toan actuator circuit for translating the output signal to actuatormovement along the at least two orthogonal axes over a digitalinterface.
 21. The method of claim 20, further comprising determining aposition of the lens relative to the image sensor along the at least twoorthogonal axes using a position sensor.
 22. The method of claim 21,further comprising providing feedback from the position sensor to theactuator circuit.
 23. The method of claim 22, further comprisingperforming a calibration routine using the feedback.
 24. The method ofclaim 20, further comprising simultaneously determining movement of thefirst camera unit and providing output to a host processor configured torun a process utilizing output from the motion sensor.
 25. The method ofclaim 24, wherein the process utilizing output from the motion sensor isa user interface function.
 26. The method of claim 24, wherein themotion sensor further comprises an accelerometer and providing theoutput to the host processor by the motion processor comprisesperforming a sensor fusion operation.
 27. The method of claim 20,wherein the actuator moves the lens along three orthogonal axes andwherein the motion sensor senses motion along three orthogonal axes,further comprising performing an autofocus (AF) process to output asignal corresponding to movement along at least one of the threeorthogonal axes in response to the determined movement of the firstcamera unit.
 28. The method of claim 27, further comprising receivingoutput from the image sensor with an image processor, wherein the AFprocess includes focus information determined by the image processor.29. The method of claim 20, further comprising processing output fromthe image sensor to assess the compensating relative movement betweenthe lens and image sensor.
 30. The method of claim 29, furthercomprising performing a calibration routine using the assessment of thecompensating relative movement.
 31. The method of claim 20, furthercomprising calibrating the motion sensor and the actuator independently.32. The method of claim 20, further comprising: providing a secondcamera unit including a lens, an image sensor and an actuator for movingthe lens relative to the image sensor along at least two orthogonalaxes; determining a compensating relative movement between the lens andthe image sensor of the second camera unit; and controlling the actuatorof the second camera unit to produce the compensating relative movementusing the digital interface.